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1 Introduction 

The TAP equations [6J for the Sherrington-Kirkpatrick model describe the quenched 
expectations of the spin variables in a large system. 

The standard SK- model has the random Hamiltonian on S^r = {-1,1} , N e N, 

N 

l<i<j<N i=l 

where a = (<ti, . . . ,<jn) £ ^N, P > 0, /i > 0, and where the , I < i < j < N, are 
i.i.d. centered Gaussian random variables with variance 1/N, defined on a probability 

space (Q, F, P) . We extend this matrix to a symmetric one, by putting gij d = gji for 

def 

i > j, and gu = 0. The quenched Gibbs measure on £jv is 

— — exp [-H N ,p,h,u 0)] , 

where Z N ,p }hjU) d = ex P [~ H N,i3,h,Lu 0)] • 

We write (-)jy « ^ for the expectation under this measure. We will often drop the 
indices N, /3, h, uj if there is no danger of confusion. We set 

def , . 

mi = (<Ti) • 



The TAP equations state that 

nii = tanh (h + P ^ 9ij m i ~ P 1 (1 - 0.) m i^j , (1-1) 

which have to be understood in a limiting sense, as N — >• oo. q = q (/3, h) is the solution 
of the equation 

q = J tanh 2 (h + fiy/qz) <j) (dz) , (1.2) 
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where (ft (dz) is the standard normal distribution. It is known that this equation has 
a unique solution q > for h > (see [I] Proposition 1.3.8). If h = 0, then q = 
is the unique solution if < 1, and there are two other (symmetric) solutions when 
f3 > 1, which are supposed to be the relevant ones. Mathematically, the validity of 
the TAP equations has only been proved in the high temperature case, i.e. when j3 is 
small, although in the physics literature, it is claimed that they are valid also at low 
temperature, but there they have many solutions, and the Gibbs expectation has to be 
taken inside "pure states". For the best mathematical results, see [1] Chap. 1.7. 

The appearance of the so-called Onsager term f3 2 (1 — q) rrii is easy to understand. 
From standard mean-field theory, one would expect an equation 



( ^ N \ 

tanh I h + /3 . 1 9ij m j J 



but one has to take into account the stochastic dependence between the random variables 
rrij and g^. In fact, it turns out that the above equation should be correct when one 

(i) 

replaces rrij by rrij where the latter is computed under a Gibbs average dropping the 

(i) 

interactions with the spin i. Therefore rrij is independent of the gn-, 1 < k < N, and 
one would get 

rrii 



tanh (h + P J2* =i gtjmf) . (1.3) 



The Onsager term is an Ito-type correction expanding the dependency of rrij on gji = g^ 

and replac 
because of 



U) 

and replacing rrij on the right hand side by rrij . The correction term is non- vanishing 
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9ii ~ 1, 



i.e. exactly for the same reason as in the Ito-correction in stochastic calculus. We omit 
the details which are explained in [3]. 

In the present paper, there are no results about SK itself. We introduce an iterative 
approximation scheme for solutions of the TAP equations which is shown to converge 
below and at the de Almayda-Thouless line, i.e. under condition ()2.ip below (see pQ). 
This line is supposed to separate the high-temperature region from the low-temperature 
one, but although the full Parisi formula for the free energy of the SK-model has been 
proved by Talagrand [5], there is no proof yet that the AT line is the correct phase 
separation line. 

The iterative scheme we propose reveals, we believe, an interesting structure of the 
dependence of the rrii on the family {gij} , even below the AT line. The main technical 
result, Proposition 12.51 is proved at all temperatures, but beyond the AT-line, it does 
not give much information. 

We finish the section by introducing some notations. 

If x,y G M. N , we write 

j N 

i=i 
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As mentioned above, we suppress N in notations as far as possible, but this parameter 
is present everywhere. 

We also define the N x iV-matrix x(g> s y, x (8) y, by 

, \ def 1 , \ def %iUj ,\ 

( x ® S y)y = [XiVj + XjVi) , x®y = — . (1.4) 

If A is an N x iV-matrix and x G R^, the vector Ax is defined in the usual way 
(interpreting vectors in R N as column matrices) . If / : R — )■ R is a function and x £ R 
we simply write / (x) for the vector obtained by applying / to the coordinates. 

g = (gij) is a Gaussian iVx iV-matrix where the g^ for i < j are independent centered 
Gaussians with variance 1/N, and where = gji, gu = 0. We will exclusively reserve 
the notation g for such a Gaussian matrix. 

We will use Z, Z', Z\, Z2, ■ ■ ■ as generic standard Gaussians. Whenever several of 
them appear in the same formula, they are assumed to be independent, without special 
mentioning. We then write E when taking expectations with respect to them. (This 
notation is simply an outflow of the abhorrence probabilists have of using integral signs, 
as John Westwater once put it). 

If {Xn} ,{Yn} are two sequences of real random variables, defined on (fl, F, P), we 
write 

X N ~Y N 

provided there exists a constant C > such that 

¥(\X N - Y N \ >t)< Cexp [-t 2 N/C] 

for all N e N, < t < 1. 

Clearly, if X N ~Y N , and X' N ~ Y^, then X N + X' N ~ Y N + Y^. 

If = \Xj ) , = [Y; ) are two sequences of random vectors in 



0. 



i<N \ J i<N 

R N , we write « Y< ff ) if 

i=i 



We will use C > as a generic positive constant, not necessarily the same at different 
occurrences. It may depend on f3,h, and on the level k of the approximation scheme 
appearing in the next section, but on nothing else, unless stated otherwise. 

In order to avoid endless repetitions of the parameters h and f3, we use the abbrevi- 
ation 

Th (z) = tanh (h + fix) . 

We always assume h ^ 0, and as there is a symmetry between the signs, we assume 
h > 0. q = q ((3, h) will exclusively be used for the unique solution of (|1.2|) . In the case 
h = 0, P > 1, there is a unique solution of (|1.2p which is positive. Proposition 12.51 is 
valid in this case, too, but this does not lead to a useful result. So, we stick to the h > 
case. 

Gaussian random variables are always assumed to be centered. 
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2 The recursive scheme for the solutions of the TAP equa- 
tions 



We recursively define a double sequences m( fc ) = j \ 
by putting 



<N, fcgN 



of random variables 



m 



(o) 



dcf 



0, m 



(1) 4ef 



y/ql, Vt < iV, 



1 here the vector with coordinates all 1, and q = q(j3, h) is the unique solution of (|1.2j) 
We define 

m (*+i) <M xh ^ gm (fc) _ p (1 _ g) m (fc-irt k > h 

k will exclusively been used to number this level of the iteration. Our main result is 
Theorem 2.1 

Assume h > 0. If (3 > is below the AT-line, i.e. if 



/3 2 £cosh- 4 (h + Py/qZ) < 1, 



(2.1) 



then 



lim lim sup E 

k,k'— >oo TV— >oo 



m 



(fc) 



m 



(fe') 



0. 



Jf there is strict inequality in \2.1\) . then there exist < A h) < 1, and C > 0, such 
that for all fc 



lim sup E 



m (*+i) _ m W 



< cx 



The theorem is a straightforward consequence of a computation of the inner products 
^mW, mOO) . "We explain that first. The actual computation of these inner products will 
be quite involved and will depend on clarifying the structural dependence of on g. 

As we assume h > 0, we have q > 0. We define a function ijj : [0, q] — > M by 

^ (t) d = £ Th ( y/tZ + v^i^'l Th (ViZ + v 7 ^ 7 *^") , 



where Z, Z', Z", as usual, are independent standard Gaussians. Remember that Th (x) = 
tanh (h + fix) . 

Let a d = ETh(^qZ) > 0. 
Lemma 2.2 

a) ifi satisfies < tp (0) = a 2 < tp (q) = q, and is strictly increasing and convex on 
[0,9]. 



b) 



V (q) = (3 2 E cosh" 4 (h + Py/Q Z ) ■ 
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Proof, ip (0) = a 2 , and tp (q) = q are evident by the definition of a, q. We compute the 
first two derivatives of tp : 



1 



ZTh' (y/tZ + Vq~^tZ') Th (VtZ + ^/q^tZ" 



-.E 



Z' Th' ( VtZ + Vq^tZ'j Th ( VtZ + Vq^Z" 



= E Th" (VtZ + Vq^tZ'^j Th (sftZ + Vq^tZ" 

+ e Th' ( Vtz + Vq^tz'^j Th' ( Vtz + Vq^tz"^ 
- e Th" (Vtz + Vq^tz'^j Th ( Vtz + Vq^tz"^ 
= e Th' f Viz + Vq=tz') Th' f Viz + Vq^tz") . 



the second equality by Gaussian partial integration. 
Differentiating once more, we get 

t//' (t) = e ( Th" (Viz + Vq^tz') Th" ( Viz + Vq^tz" 



In both expressions, we can first integrate out Z' , Z", getting 

2 



V 



/oo p oo 
/ Th'(7te + Vq=ty)<t>{dy) 
-oo J —oo 



4> {dx) > 0, 



and the similar expression for t/j" with Th' replaced by Th" . So, we see that if) is in- 
creasing and convex. Furthermore, as 

Th' (a?) = P tanh' (/3a? + h) = /3 (l - tanh 2 (#c + /i)) 

/3 



we get 



cosh 2 {fix + h) ' 
^ (q) = E Th' (^Z) 2 = /3 2 E cosh" 4 (ft + /3y/qZ) . 



Corollary 2.3 

If i2.1\) is satisfied, then q is the only fixed point ofip in the interval [0, q] . If i2.1\) is not 
satisfied then there is a unique fixed point of ip (t) = t inside the interval (0, q) . 



We define sequences {pk}k>i > ilk}k>i recursively by 71 = f a, p\ d = 71^/9, and for 



k > 2 



Pk d = tp (pk-l) 



7fc 



def 
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where 



Lemma 2.4 

a) For all k e N 



m 

„2 def \ -> 2 t^2 def 



fc-1 



< Pk <q- 



b) If (12.1)) is satisfied, then 



lim = g, lim r| = g. 

k— >-oo fe— >oo 



c) If there is strict inequality in (12. 1|) , then arid converge to g exponentially 
fast. 

Proof, a) pk < q for all k is evident. 

We prove by induction on k that pk > r|_ r For k = 1, as p\ = ^fiy/q, the statement 
follows. 

Assume that it is true for k. Then 



Pk ~ / — 

7fc = ; < a Pk ~ T z 



fc-1' 

q - rti 

i.e. jOfe > r|. As Pk+i > pk, the statement follows. 

b) Evidently lim/ c _ i . 00 pk = q if ()2.ip is satisfied. The sequence {T^} is increasing and 

bounded (by g). If C = f um fc-»oo F| < q, then lim^oo jk = Vq ~ C > 0, a contradiction 
to the boundedness of {i^} . 

c) Linearization of ip around q easily shows that the convergence is exponentially fast 
if fj/ (g) < 1. ■ 

Remark that by a) of the above lemma, one has 7^ > for all k. 
Let IIj be the orthogonal projection in R , with respect to the inner product (-, •) , 
onto span (m^, . . . , m^)) . We set 

M (k,j) dj m (k) _ jj . f(k)\ ( ■ < k) {2 2) 



and 

Let 



M (k) M (*,k-D. (2 .3) 



def 



M( fc ) 

|mW|| 

def 



(2.4) 



if ||M( fc )|| 7^ 0. In case m( fe ) £ span (m^, . . . , m^" 1 )) , we define = 1, to have 
it defined everywhere, but we will see that this happens only with exponentially small 
probability. Remark that (f>^ = 1. 
The key result is: 
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Proposition 2.5 

For all k G N 



and for 1 < j < k 



(k) 



m 



m (i) m (fc) 



(2.5) 

(2.6) 
(2.7) 



Proof of Theorem 12.11 from Proposition 12.51 As the variables m( fc ) are bounded, 
(|2.5p implies 



lim E 

N^oo 



m 



(*) 



and similarly for the other statements. 



E 



m 



(fc+i) 



m 



E 



m 



(k) 



E 



m 



q, 



2E / mW, m^- 1 ) 



Taking the — )• oo limit, using Proposition 12.51 this converges to 2g — 2p^_\. From 
Lemma 12.41 the claim follows. ■ 



Remark 2.6 

Proposition 12.51 is true for all temperatures. However, beyond the AT-line, it does not 
give much information on the behavior of the m\ k > for large k. It would be very interesting 
to know if these iterates satisfy some structural properties beyond the AT-line. 

The main task is to prove the Proposition 12.51 It follows by an involved induction 
argument. We first remark that (|2.7p is a consequence of (|2.5[) and (|2,6p . 

If J G N let COND (J) be the statement that ([23]) and $Mj§ hold for k < J. COND (1) 
is evidently true. 

COND (J) implies that for all k < J, we have with 



that 



If we put 



then 



g-rti/2>0 
M( fc )|| <,5 fc ) < C k exp[-N/C k ] 

Aj d = f n{||MW||>4}, 



k=l 



>(Aj)>l-Cjexp[-N/Cj} 



(2.8 



(2.9) 



Evidently, all variables cp ' are bounded by a constant on Aj, if k < J. The constant 
may depend on J, of course. The are bounded by 1 everywhere. 
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3 Iterative modifications of the interaction variables 

Let Q be a sub-a-field of T, and y = (yij) 1<:i j <N be a random matrix. We are only 
interested in the case where y is symmetric and on the diagonal, but this is not 
important for the moment. We assume that y is jointly Gaussian, conditioned on Q, i.e. 
there is a positive semidefinite iV 2 x iV 2 - Q-m.b. matrix T such that 



(We do not assume that y is Gaussian, unconditionally). Consider a ^-measurable 
random vector x, and the linear space of random variables 



We consider the linear projection ire (y) °f y onto C, which is defined to be the unique 
matrix with components TT£ (t/ij) in C which satisfy 



As y is assumed to be conditionally Gaussian, given Q, it follows that y — ire (y) is 
conditionally independent of the variables in C, given Q. 
If y is symmetric, then clearly ttc (y) is symmetric, too. 

Remark 3.1 

If X is a ^-measurable random variable then yX is conditionally Gaussian as well and 



Remark also that 

(y - tt£ (y)) x = yx-vr £ (yx) = 0, (3.1) 

as yx G C. 

Using this construction, we define a sequence g^ k \ k > 1 of matrices, and a sequence 

{Fk} of sub-d-fields of J 7 , starting with = f g, and T-\ = To = {0,0}). The 
construction is done in such a way that 

(CI) g( fc ) is conditionally Gaussian, given 

(C2) m( fc ), M.( k \ and are J 7 fc „i-measurable 

Using that we define 



E 



(exp [i^2 k .tkjVkj 0) 




®({y ij -Trc(yi j )}U\g) = 0, VC/g£. 



tt£ (y) x = ttc (yX) . 




with 
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i.e. we perform the above construction with Q = J~k-i and x = M( fe ). 
Furthermore, we define 

^. + i = ^(^,e (fc+1) ), 

where 

£(*) A g g(*)0(*). 

In order that the construction is well defined, we have to inductively prove the 
properties (CI) and (C2). We actually prove a condition which is stronger than (CI): 

(CI') Conditionally on J-k-2-, g^ is Gaussian, and conditionally independent of J~k-i- 

(CI') implies that is conditionally Gaussian, given J-k-i, and the conditional 
law, given Tk-i, is the same as given 7^-2- 

Inductive proof of (CI') and (C2). The case k = 1 is trivial. We first prove (C2) 
for k > 2, using (CI'), (C2) up to k - 1. We claim that 

m« = Th (g(*-i)M(M + R( fe - 2 )) , (3.2) 

where R^ -2 -* stands for a generic J-fc_2-measurable random variable, not necessarily the 
same at different occurrences. 

As g( fc - 1 )M( fc - 1 ) = ||M( fc-1 >|| ^ fc - 1 ), and M^ 1 ) is J" fc _ 2 -measurable, by the induc- 
tion hypothesis, it follows from (|3.2p that m( fc ) is J-fc_i-measurable The statements for 
are then trivial consequences. 

We therefore have to prove (|3.2p . We prove by induction on j that 

m« = Th (g^M^- 1 ^ 1 ) + R( fc - 2 )) . (3.3) 

The case j ' = 1 follows from the definition of mS k \ and the case j = k — 1 is ()3.2p . 

Assume that (|3.3|) is true for j < k— 1. We replace g^^ by g(-? +1 ) through the recursive 
definition 

m (ft) = Th rgU+DM^- 1 ^'- 1 ) + n C] ( g W>) Mf*- 1 ^'- 1 ) + R (fc " 2) ) 
= Th (g^M^^+R^- 2 )), 

as 7T£j (g^) is -Fj-measurable and therefore ncj (g^) MA fc— ^ is .F/^-measurable 
Using ()3.ip . one gets gtf+^Mw = 0, and therefore 

g (i+i) M (fc-i,i-i) = g (i+i) M (fc-ij)_ 

This proves (j3.2[) . and therefore (C2) for k. 
We next prove (CI') for k. 

gW = g(*-i) _ ^ ( g (*-i)) . 
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We condition on J-k-2- By (C2), M^ -1 ) is J-/ c _2- measura t ) le As g^" 1 ), conditioned on 
Tk-s, is Gaussian, and independent of Tk-2, it has the same distribution also conditioned 
on J-fc_2- By the construction of g( k \ this variable is, conditioned on T^-i-, independent 
of .Ffc-ij and conditionally Gaussian. ■ 

Lemma 3.2 

For m < k, one has 

g (*)0(m) = Q 

Proof. The proof is by induction on k. For k = 1, there is nothing to prove. 

Assume that the statement is proved up to k. We want to prove g( fc+1 )(^( m ) = for 
m < k. The case m = k is covered by (|3.ip . For m < k, it follows by Remark 13.11 as 
^(m) j g j^_ 1 _measurable, that 

*A (g (fe) ) <P {m) = *C k (g (fc) (m) ) , 

and therefore 




as g( k )(f>( m ') = by the symmetry of g( fc ) and the induction hypothesis. ■ 

Lemma 3.3 

If m < k, then 

E^ (m) = °- 

i 

Proof. 

= E^ £) E4? ) ^ m) 
= E^ fe) E^ m) = o 

for m < k, by the previous lemma. ■ 

4 Computation of the conditional covariances of g( fc ). 

We introduce some more notations. 

We write (N~ r ) for a generic J-fc-measurable random variable X which satisfies 

P (N r X >K)<C exp [-N/C] , 
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for some K > 0. The constants C, K > here may depend on h, (3, and the level k, and 
on the formula where they appear, but on nothing else, in particular not on N, and any 
further indices. For instance, if we write 

Xij = Yij + O k (N~ 5 ) , 

we mean that there exists C (/3, h,k) , K (/3, h,k) > with 

supF (iV 5 \Xij - Yij\ >K)< Cexp [-N/C] . 

ij 

Furthermore, in such a case, it is tacitly assumed that — Y{j are .7-fc-measurable 
Evidently, if X,Y are O k {N- r ) , then X + Y is O k (N- r ) , and if X is O k (N- r ) , 

and Y is O k {N" s ) , then XY is O k (N~ r - S ) . 

We write for the conditional expectation, given T k . 
We will finally prove the validity of the following relations: 

Efc^f = ^ + Ofc-2 (N- 2 ) . (4.1) 

fc— 1 Am) Am) 

^gfgf = - £ ^Jr~ + O k -2 (iV" 3 ) , V*^i,i (4.2) 

m=l 
(fc) 

a; 



where 



with 



Efc-tfg ^ = J?+ O fc _ 2 (iV- 4 ) , if *} n {i, i} = (4.3) 

fc-i 



(fc) \ ^ \ ^ .(fc) Am) 
m=l Ac{i,j,s,t} 



, (m) def 

Va = 

ueA 



n ^ m) - 



(fc) 

The Ai^ a are real numbers, not random variables, which depend on A only through the 
type of subset which is taken. For instance, there is only one number (for every m, k) if 
all four indices are taken. 

The main result of this section is: 

Proposition 4.1 

Let J G N, assume COND (J) , and assume the validity of <POp - ifO]) hold for k < J. 
Then they hold for k = J + 1. 

The main point with assuming COND (J) is ([2.9p . On Aj, the variables cf>^ are 
bounded for k < J. 

Lemma 4.2 

Assume g3P - (B3P for k = J, and (230 . Then 
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a) 

= 1 + Oj-i (iV- 1 ) . (4.4) 

b) 

Kj-i^ = ±<P4>? - ± £ 4 ] 4 ] + Oj-i (iV- 2 ) (4.5) 

r=l 

,(J) 

Ej-tf^ = \ + Oj-j (iV- 2 ) • (4.6) 

d) For s ^= i,j 

rra=l m=l 

Proof, a) As (f>W) is J-j_i-measurable, and is independent of J-j-i, conditionally 
on Jj_2, we get 

= E #V! j> e,-, (W) = E 4M r %-i 
=£^ 2 e,- 2 01/ )2 ) + £ ^Vi J) E,- 2 

Using gU, ([I2D, and the boundedness of the <£'s on A j, and TV" 1 £\ ^ (J)2 = 1, E* 4^4^ 
for m < J, we get 

Ej-i^ 2 = 1 + Oj-i (iV" 1 ) . 

b) e j _ 1 ^= e ^^(^ 

We split the sum over (s, t) into the one summand s = j, t = i, in ^4 = {(s, s) : s ^ i, j} , -B 
{(j,t) : C = {(a,*) :s^t,j}, and D = {(a, t) : {a, t} n {i, j} = 0} . The one 

summand s = j,t = i gives <pf (j>j /N + Oj-i (^V~ 2 ) • 

r j-i A m )A m ) 1 

E = E = E *S J 12 - E ^4^- + o M 



TV 

m=l 
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E = E tfVS**- («W 



B t&,j 



E 



J - 1 J. m )J- m ) 



m=l 



Because {4>^ J \ (j>( m ty = for m < J, this is seen to be Oj_i (iV 2 ) . The same applies to 

It remains to consider the last part YId ■ Here we have to use the expression for 
Ej_2 (dipyit^ where {i,j} n {s,i} = given in c) of Theorem 14.11 



E <t> ( M J) 

s,t:{s,t}n{i,j}=<A 



1 - 7 " 1 

V" \( J ) 

m=l Ac{ij',s,t} 
J-l 



+ Oj-2 (iV" 



= ^s E «W£ E O^+^M- 

s,t:{s,t}n{i,i}=0 m=l Ac{i,j,s,t} 

Take e.g. ^4 = s} . Then = A^ 3 with no further dependence of this number 
on s. So we get for this part for any summand on m with m < J 



— \ {J) 
jy3 m,3 



E ^M^ti 



m) Am) 



:,t:{s,t}n{i,j}=0 



Using again (^(f)^ J \ (j)^ m ^ = 0, we get that this is Oj—\ [N 2 ) . This applies in the same 
way to all the parts. Therefore b) follows, 
c) 



v n (J)A-J)_srAJ v A J )A J 



+ £ 



m=l 



. ZJ_ , n> /ikt-2\ 

N 



N 2 



+ Oj_i (N- 



due to the orthogonality of the (f)( m ) . 
d) 



3 v% 3 v S j 
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due again to (|4.3p . We therefore get 

J-i 



i J )c(J) 



1 



m=l 



+ Oj_! (iV- 3 ) 



Lemma 4.3 

We assume the same as in Lemma\4.2[ Put 



J J h( J ) _u A J h^ 1 N 



iV 



4 TV 2 



r=l 



Then 



.9 



(J+i) _ -(J) 



where the Tj-i-measurable coefficients x^ s satisfy 



E 



0, Vi,j, Vm < J, 



(4.8) 



(4.9) 



with 



s5,; = Oj_i(JV-^) J se{i,i}, 



(■/) 



Oj_i(j\r d ), ^{y}, 



Proof. The existence of J-j-i-measurable coefficients x^ comes from linear algebra. 



Remark that 

E «W = E *?W = E [E. 4 



0. 



Therefore, we can replace the x[fl by 



m=l 



which satisfy the desired property ()4.9p . 

We keep i,. 
is that for all t 



We keep i,j fixed for the moment and write x s for The requirement for them 



M^-E.^'K' 

From Lemma 14.31 we get 



0. 



^J-i(^M J) )=Oj- 1 (iv- 2 ) 
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and the same for y9ij\j J j • For t ^ , we have 

AJ) J-i 

m=l 

m=l ) 



AJ) J- 1 

m=l 


4 J> j 




A< J > 


JV ] 






*J° j 






JV 1 







m=l 



AT2 



1 



r^f I m=l 



Due to the orthonormality of the eft, one gets 

^0^)0^ = 0^ (iv- 1 ). 

So we get 

We write for the moment yt = f Ej-i \9ip€tP) ■ Fhe equations for {x s } are 



Writing r,j for the Oj-\ (N 2 ) error term in (j4.5|) . and for j = i, the Oj_i (N x ) error 
term in (14.41). we arrive at 



E^ \^ J) 4>[ J) - ^ E ^V{ W) + r J + x t (1 + r«) = 

s^t I m=l J 

In the first summand, we sum now over all s, remarking that we have assumed that 
x s (j)i m ^ = for m < J. The error for not summing over the single t can be incorpo- 
rated into rtt- We therefore arrive at 



xt + 4>\ J) jz e x s ( i ){ s ) + E Xsr - st = yt - 
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Write <I> for the matrix yN l (fr i (fr- J and R for (rjj) . Then we have to invert the 
matrix (7 + $ + R) . Remark that (7 + = 1-^/2. Therefore 

(/ - $/2) (I + $ + R) = I+(I- $/2) R. 

The right hand side, we can develop as a Neumann series: 

(/ + $ + Ry 1 (/ + $) = (/ + (/- $/2) Ry 1 

= /-(/- $/2) i? + [(/ - $/2) R} 2 

v+*+«>- l = i-l-('-l)*(i-?) + -- 

As ($y)j = Oj-i j w e get the desired conclusion. ■ 

Proof of Proposition 14.11 

^J-l [9ij 9st ) ~ ^J-l [9ij 9st ) + Z^ X st,u^J-l [€u 9ij ) 

u 

+E^!>- 1 (^ ) ) (4-io) 

u 

u,v 

The summands involving the all only give contributions which enter the Oj_i-terms. 
Take for instance s = j, t ^ In that case, the claimed Oj_i-term is Oj-\ (A~ 3 ) . 
In the last summand of (|4.10p . there is one summand, namely u = v = j, where the x^ 
are Oj-i (A~ 2 ) , so this summand is only Oj_i (A~ 4 ) ■ 

E 4>« (<&%») = (W) + 43^- ( W) 

+ xgE,_ 1 ( e «^) 

+ £ (f«"^») . 

From Lemma 1331 we get E j_i Ui^g^ = Oj-i (iV _1 ) for u G {», j} , and O j_i (iV^ 2 ) 
otherwise. So the above sum gives 

Oj_! (AT- 3 ) Oj_! (A- 1 ) + Oj_! (A- 2 ) Oj_! (A" 1 ) 

+ Oj_! (A" 2 ) Oj_i (A- 2 ) + NOj^ (N- 3 ) Oj-x {N~ 2 ) 

= Oj_i (A- 3 ) . 

The other summands behave similarly. The third and fourth summand in (|4.10p behave 
similarly. 
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As another case, take {i,j} Pi {s,t} = 0, where we have to get Oj—x (N 4 ) for the 
second to fourth summand in (|4.10p . 

(J) 



E + E + E 

u=i,j U=S,t U$:{i,j,S,t} 

= Oj-x (N- 3 ) Oj-x {N- 1 ) + Oj_! (AT- 2 ) Oj_! (iV- 2 ) 
+ iVOj^! (N- 3 ) Oj- X (N~ 2 ) 
= Oj„x {N~ 4 ) . 

E«^(^ J) )= E + E +E E 

u,v u=v£{i,j,s,t} u=v^{i,j,s,t} u&{i,j} v £{s,t} 

+ £ £ + £ £ 
+£ £ £ 

= Oj_i (iV- 5 ) + JVOj-j (A^ 6 ) 
+ Oj_! (iV- 2 ) Oj_! (iV" 2 ) Oj_! (iV- 1 ) 
+ NOj-! (N- 2 ) Oj-x (N- 3 ) Oj-x (N- 1 ) 
+ NOj-x (N- 2 ) Oj-x (N- 3 ) Oj-x {N- 1 ) 
+ N 2 Oj-x {N- 3 ) Oj-x {N- 3 ) Oj_i (A^ 1 ) 
= Oj_! (iV- 5 ) , 

which is better than required. 

It therefore remains to investigate E j_i ( g^ +1 ^ 



a) 



Ej-i a 



} (J)2 



E 



j-i 



(j) gj 4j +0j gj gj V^-a( j )^ 



N 



(J)c(J) 



Using Lemma \4.2\ one easily gets that anything except Ej_i (<?1^ 2 ) is Oj-i (iV 2 ) 



Ej_i (glpj = Ej_2 (^dip 2 ^ from the conditional independence of g( J ) of Tj-x, given 
Fj-2- So the claim follows. 



b) 



rj) <Pi 4j + gj Sj 



+ 



N 2 



E<^ J) 



17 



We write m x n for the summand, we get by multiplying the m-th summand in the 
first bracket with the n-th in the second. By induction hypothesis, we get 

ixi = -E^^ + °^(^ 3 )- 



m=l 



In the 1 x 2-term, only the multiplication of g\'p with £^ counts, the other part giving 
Oj-2 {N' 3 ) . Therefore 

AJ) 

2x1 gives the same. In 2 x 2, again only the matching of £j with £^ counts, so we 
get 



2 x s 



<>\ '<t>\ 



N 2 



+ Oj-i (N- 



The other parts are easily seen to give Oj_i (N ) . We we have proved that 



Ej-i (9ij'9jt 



c) We have here {i,j} H {s,t} 



J Am) Am) 



m=l 



N 2 



AJ)~(J) 



iV 



+ 



N 2 



E^ J) 



x 5, 



(J) 9s t, t + ( Pt 4s , <Ps <p t 



St 



N 



+ 



N 2 



The 1 x 1,1 x 2,2 x 1, and 2 x 2-terms are clearly of the desired form, either from 
induction hypothesis or Lemma 14.21 



For u = i we get for the expectation cj)[ J ^ (j)^ / N + Oj_i (iV 2 ) , so this is of the desired 
form. The same applies to u = j. It therefore remains 

n '<t>\ ,f./w ( aj)aj) 



N 2 



J-l 



N 2 



(m) 
m=l 



+ J _ 1 (iV" 4 ). 
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As J2 U 
similarly. 



0, the whole expression is Oj-i [N J . The other cases are handled 



5 Proof of Proposition 12.51 

We assume COND (J) , and flU]) - (gSJ) for k < J. By Proposition O of the last 
section, this implies (|4.ip - (|4.3p for < J + 1. Using this, we prove now f)2.5j) and (|2.6p 
for fc = J + 1, so that we have proved COND ( J + 1) . Having achieved this, the proof 
of Proposition 12.51 is complete. 

For j < k, define = 0, X^') d = Et=i7^ (t) > and x(fc,fe) = f £t=i + 

Remark that under COND (J) 



m 



(fc) _ x (fe,fc)_ 



(5.1) 



for k < J. Indeed 



m(tL C( m( ^ H )^ 

^ — 'm=l \ / 
* — 'm=l 



From g > r|_ l7 by (|2.5p and ([2.6p for k < J, and the fact that the <jr- are uniformly 
bounded on Aj, we have 



m (H_^ m (^H^) 



So the claim (|5.ip follows. 
We define for 1 < s < k 



m 



(M 



def 



Th (gC-JM^- 1 -- 1 ) + Y, S t= l 7t£ W + (1 - g) {X 



m 



(fc-2) 



}) 



Remark that by Lemma l3T2"l we have g( s )M( fc 1,3 x ) = g( s )m( fc ). Evidently 



m (M) _ m (fe) 



and we define 



m 



(1) 4ef (l) 



m«=^l, mW^m^- 1 ), k > 2. 
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By dSHD 



A (fc) = Th fgf*-V-D + ^ 2 7^ (t) ) 

= Th f g (*-%(*-i) + 

M(fc -l)| e ( fe -l) + ^-2 7 ^ (t) 



Th 



The key result of our paper is 
Proposition 5.1 

holds for all k. 

This proposition is correct for all f3. The key point with (|2.ip is that the first summand 



(5.2) 



||]y[(fe i) disappears for k — > oo as ||M( fe ~ — r|_ 2 , so that for large k, 

m( fc ) stabilizes to Th ( Y^tZi 7t£^ ) , but above the AT-line q — T\_ 2 does not converge 



to 0. Therefore, above the AT-line, in every iteration, new conditionally independent 
contributions appear. 

The above proposition is proved by showing that COND (J) implies 



m (J+1) ps m (J+1) . 



(5.3) 



As COND (J) implies trivially COND (J 7 ) for J' < J, it is then clear that COND (J) 
implies ~ rh.( k ) for all k < J + 1. As the are uniformly bounded by 1, we get 
from that 



m (J+i) m (3) 



for all i < J + 1. We will then prove (Lemma 15 .3j) that 
for j < J, and 

2 

This will prove COND ( J + 1) , and therefore, this will have finished the whole induction 
procedure. 

Together with proving (|5.3p . we also show 



(5.4) 



for fc = J + 1 which is not evident from (j5.3j) as the are not bounded. 
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Lemma 5.2 

Assume the validity of lf£5]) - lf2T|) and {E§ for k < J. Then for s = 1, . . . , J - 1 



In particular, it follows 



m (J+l,s) ~ m (^+ 1 . s + 1 ). 



m (J+l) ra m (^+l)_ 



Furthermore ( 15.4)) holds for k = J + 1 . 

Proof. We prove by induction on s, 1 < s < J — 1, that 



m (J+l,«) ~ m (J+i,«+i) 



and 



m (j+i, s )\ „ ^ (m ) )m (j +1 , s+ i)\ ^ m < j_ 



We have 

g (*+l) = g (s) _ 0s + 0(«)\ ^(«) g, + C W 

where 

>) _ V r (s) ^ 

r 

see Lemma 14.31 Therefore 

m (J+i,s) = Xh ( g (.+l) m (J) + y+/3 (1 _ ?) jxC- 7 " 1 '- 1 ) - m^}) 

where 

y ^ (<^\ m^) e (s) + (&\ m( J )) 0« + (<^>, m^) (<^\ £ (J) ) W + c^m 
We write 

y« d ^ (^),m( J ))e (s) + (e (s) ,m( J )) 0« + ^ s \m^) (^,£«) 



y(3)^ 7 ^W + /^) >m (. 



y (4)d = f 7,e {s) + (e (s) ,m (J) )0 (s) , 



,(5) 



def 



= 7 ,e (s) + /3(i-g)7^ {s) , 

and then set ad hoc 

M (o) d jf m ( J+1 ' s ), 
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and define // n ) where y is replaced by y^ n \ n = 1, . . . , 5. Remark that 



We will prove 
and 



M (5) = m (J+l,-+l). 



(n-1) ~ „(n) 



M W n = l,...,5, 



$H^M) ~ fN,^)), n = l,...,5. 



(5.7) 
(5.8) 



which prove the desired induction in s. 

To switch from to /x^, we observe that by the estimates of Lemma 14.31 one has 



< O.-1 (1) 



Therefore 



a e 

TV ^ 



(o) (i) 



< 



N 2 ' — 'j 

Os-l (1) \ - Us) 



N 2 



and 



£ E M"* K» ^!")|^(sE |<W | + £ E If."'" I * E N*> I ) 

By choosing if large enough, we get for 1/y/N < t < 1 by Corollary IA.2l a) 

p E h (0) - /4 1} | > *) < p (§ E i 1^1 (o-i (i) > *o 



< Cexp [-iV/C] < Cexp [-Ai 2 /C] • 

For t < 1/y/N, the bound is trivial anyway. This proves (|5.7p for n = 1. (|5.8p follows in 
the same way using Corollary IA.2I b) . 



at 4^ I M 



< c 

< c 



on ^4 j . (|5.7p for n = 2 then follows from Corollary IA.2I c) . As for (|5.8p , we remark that 



^E|f 



< c 



0W,e^)||(0w,e (m) ) 



We can then again use Corollary IA.2I c) remarking that exp [— Nt/C] < exp [— A^/C] 
for t < 1. 



^EK'-mP 1 



< c 



^- (/) )-7 S |^E|e, 
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(|5.7|) for n = 3 follows from the induction hypothesis (|2,7p . and Corollary IA.2I a) . Simi- 
larly with (|5.8p but here, one has to use part b) of Corollary IA.2I 



lvL (3) -/i (4) 



< c 



/ 



on Afc, and one uses the induction hypothesis ()5,4p for J to get (j5.7j) for n = 4. Remark 
that actually, one has a bound uniform in i : 



(3) (4) 



< C 



Therefore, one also gets (|5.8p using Corollary IA.21 Up to now, we have obtained 

m (J+l,s) ra Th ( g («) M (*-l.«-l) + £^ 7 ^(t) 

+ /3 (1 - q) /x^" 1 '*- 1 ) - m^- 1 )} ) 



and 



~ (e (m) , Th (gWM^ 1 -- 1 ) + Y^ t=1 



(t) 



+ m (J) ) ^ W + /3 (1 - 9) {X^" 1 '- 1 ) - m^- 1 )} ) 
By Lemma 15.31 a) below, we have 



for s < J — 1 



/3 (1 - q) Jq - T 2 j_ 2 for s = J - 1 



(5.9) 



and we can therefore replace (^ s \ mO\ ^( s ) on the right hand side, by (3 (1 — g) 7 S ( - S * ) 
for s < J — 1, or (3 (1 — q) J q — T 2 J _ 2 (j>^ J ~ 1 ^ for s = J — 1, which is the same as replacing 



j^(J i,s i) D y ^(J l,s)_ Therefore, the lemma is proved. 

Lemma 5.3 

We assume COND(J). 



a) 



b) 



0(1-?) 7- 



for s < J — 1 



/ 



/3 (1 - q) Jq - T 2 j_ 2 for s = J - 1 ' 



for j < J, and 



m 0/+i) m 0') 



m( J+1 ),m( J+1 )) 



23 



Proof, a) Consider first the case s = J — 1. 



J-2 



(t) 



N 



N H AT 

(J-iUO 
' m * at 



1 



i=l 



i, E «, (J - 1) Th (||m<-'-')|| <f -« + ^;_; 2 7t5 f ») . 

1=1 ^ ' 



We condition on Tj—%. Then is conditionally Gaussian with covariances given in 

Lemma 14.21 a), b). We can therefore apply Lemma IA.3I which gives, conditionally on 
Tj-%, on an event -Bj_2 G which has probability > 1 — C exp [-N/C] , 



N 



i=l 



i=i v 7 



1=1 



1 - E Th 2 



i=1 L V / 

Applying now Lemma lA.31 successively to ^ J ~ 2 \^ J ~ 2 \ ... ; we get 

n AT , 



j=l 



i — e 1 Th 2 iyq — ra^Zj.! + E t=1 



= Py/q-t 2 J _ 1 (l-q). 

The case s < J — 1 uses a minor modification of the argument. One first uses Lemma 
IA.3I successively to get 



.at , N 

"'< ^ .V 

i=i i=i 



M (J-D 



M 



(J-i) 



~ £Z S Th 



^-i + E'~ 2 +1 + + ^ 

Z/-1 + 2^ t=1 



(t) 



1 - £Th" 



M(- / - 1 )|z J _ 1 + ^ =i 2 7t Z i 



/?7, (1 - q) ■ 



b) This also comes with a modification of the reasoning in a). 
Assume first j < J. 



1 N 1 N / T 1 \ 

^E-S J+1) -? } - ^£ W M Zj+ E^ 

i=l i=l ^ ' 
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In the case j = J + 1, the outcome is similar, one only has to replace the second factor 
byTh(||M(- 7 )||Z J + Ei 7 =i 1 7^ ( ' ) )- 

The next observation is that by the induction hypothesis, one can replace llM^I! 



by \/ q — r j_ 1 and we get 

N 



i=l i=l 



N / v 

^E[™(v/^^ + E;:>r) 



Th(|M«-')|e- ii +E«:;7.f. ( ")] 



in the j < J case, and 
A' 



1 - (J+l)2 



1 N / \ 



The important point is that the factor before Zj is replaced by a constant, which 
is due to the induction hypothesis. We can now proceed in the same way with £ (J_1) , 
applying again Lemma |A.3[ conditioned on and the induction hypothesis. The 

final outcome is 



N 



-Y 



m 



rhr ~ E 



Th 



i=l 



T2 J-1 Z J + Y^j ^ rZr + /ZUl > Z < 



3-1 



xTh(^-r2_ 1 z, + ^ 7r z, 



in the case j < J, and 

N 



-T. 



N 



m 



(J+l)2 



i=l 



For the latter case, the right hand side is simply q. For the case j < J, we can rewrite 
the expression on the right hand side as 



e Th (y g - v)_ x z" + 7i _ 1 z' + r,_ 2 zj Th 

We represent 



rf_ 2 z' + r,_ 2 z 



(5.10) 



q - T 2 Z" + 7i _ 1 Z / = aZi + 6Z 2 



g - r?_ 2 Z' = aZi + 6Z 3 . 



Solving, we get a 2 + b 2 = q — r|_ 2 , and 



7i-iV9-rj_ 2 . 
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Using this, we get that (|5.1Up equals 

ETh {Tj^Z + aZ x + bZ 2 ) Th {Tj_ 2 + aZ x + bZ 2 ) = if, (rj_ 2 + a 2 ) . 

V)_ 2 + a 2 = r 2 _ 2 + 7i-i^-r 2 _ 2 = pj-i. 
Therefore, for j < J, we get 

1 N 

]y E ™» - ^ (Pi-i) = Pi- 

i=l 



A Appendix 

Lemma A.l 

Let C = (Ci)i=i at be Gaussian vectors with sup N A E (£ 2 ) < oo, and sup Ni ^.j N |E (CiCj)l < 
oo. Then there exist K, C > such that 

and 

p E^ ^ 2 ^ K ) ^ Cex p t-w • ( A - 2 ) 

Proof. We can multiply the Q by a fixed positive real number. Therefore, we may 

assume that sup^-^y N |E (CiCi)l — 1/4, supjy j E (C 2 ) < 1. Put on = 1 — E (C 2 ) , and 
choose independent Gaussians U{ with EC/ 2 = a». If we prove the statements (jA.ip and 
(|A.2[) for the sequence {£j + Lj} , then it follows for the Q itself, simply because (jA.lj) and 
(jA3j) hold for the Lj. Therefore we may assume that E (C 2 ) = 1, and |E (QCj)\ < l/4iV 
for i 7^ j. Write S for the covariance matrix of {d} . £ = I + e, where |ejj| < 1/AN. 
Taking the symmetric square root 

I + a = Vl + e, 

then supjj<jy |ajj| < C/N. Therefore, we can represent the d as 

(i = Zi + /^ijZj 

3 

where the Zi are i.i.d. standard Gaussians. Then 

p E^ I6i * K ) * F E^ i^i * ^/ 2 ) + F EL i^i ^ • 

By choosing X appropriate, we get the desired estimate. 
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To prove (|A.2p . we use the same representation. As 

<-E JV Z ? + -(J2 N i*iY 

and 

P (^Ef =1 Z ^^) <^exp[-iV/C] 
for large enough K, we get the desired conclusion. ■ 

Corollary A. 2 

Assume COND (J) and k < J. 

a) For any m < k there exist C, K > such that 

P 1^ | > ^) < Cexp [-A^/C] . 

b) For any m,l, there exist C, K > such that 

P (jf E, |^°| — — Cexp [-N/C] . 

c) IfYi are J- m -i-measurahle with 

P (su Pi \Yi\ >K)< Cexp [-N/C] 

for some K, then 

Y) > i) < Cexp [-t 2 N/C] , t < 1. 



Proof. Conditioned on .F m — i, is Gaussian with covariances given by Lemma 

4.21 On T m -\ -measurable events Bn with P (-Bat) > 1 — C exp [— N/C] , the variables 
appearing in this lemma on the right hand sides are appropriately bounded. So, on Bn, 
the Q are Gaussians which satisfy the conditions of the previous lemma. So a) follows 
from that lemma. For b), we estimate 



ly U (m) e (0 <J-y e (m) \ /-V e 



; (I)2 



so that we see that it suffices to consider I = m. Then we apply the lemma, part b). 

As for c), we have that the conditional distribution of y/~N Y), given F m -\, is 

Gaussian, with bounded variance. So the statement follows. ■ 
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Lemma A. 3 

Let \rii N ^\ , be Gaussian vectors with cr-f^ = n^ N \ We assume that for some 

I 1 ) i<N 3 3 

sequence (In > with log \x n being bounded, one has 



<C/N, 



and there are vectors j \ , \ yj r ' N ^ \ , m fixed, which are bounded in all 

I 1 ) i<N I 1 ) i<N, r<m 

indices, such that 



sup N z 



(AO _ X i X j 

13 N 



(N,r) (N,r) 

sp m Vi Vj 



N 



< oo. 



Let also Fnj, i < N, be functions 
in N, i. Then 

N 



I, which are bounded and Lipshitz, uniformly 



N N 

^EM^^E^(v^)- 



TV 

i=i i=i 
Proof. We leave out TV in notations, as often as possible. Consider 



m (r) 

Vi 



— i vTv 

r=l v 



z> 



The constant if > will be specified below. Then 



1 1 i\ 

N E i=1 Fn ^ {r]i) ~ N E i=1 F ^ fa) 



m 1 /tt AT 



r=l 



i=l 



where L is a bound on the Lipshitz constants for the Fnj, and c is a bound of the 
As _ 

P {\Z r \ > tVw) < Cexp [-t 2 N/C] , 



(r) 

(A.3) 



we get 



p (jf Ef =1 131 ^ ^ Cex p H 2Ar / c ] . 
a? E Fjv >* (^-ivE^ fa)- 



i=i 



m (r)2 „ 
r=l 



£ {vWj) 



N 



+ nj, i / j- 
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where 



X def 

Oi = (Tii — UN, 



def 



X i X j 



— (Jij 



Em 
r=l 



m Ui Uj 



N 



We choose K large enough such that the N x A^-matrix V which is (r^) off diagonal, 
and 

' (r)2 K x\ . 



Em 
r=l 



2/ 



iV N N 

on the diagonal is positive definite. This is possible as < CN~ 2 . 
Let {t/j} be a Gaussian matrix with covariance matrix T. Then 

^/JlNZi + -^=# + Ui 
V Jv 



has the same distribution as {r/-} . Here we assume that {Ui} is independent of the Z's. 
So, we assume that the 77^ are presented in this way. 



1 /A 1 r, / , rr 

N E i=1 F ^ W " iv E =1 F7V - 1 



< CL 



1^1 



1 * 



A A 



1=1 



We can apply Lemma lA.ll to the vector ( \/~NUi 
on the right-hand side, obtaining 



Ki<N 



and ()A.3p to the first summand 



^ EL (vt^^) * ^ EL > 



follows by standard Gaussian isoperimetry (see e.g. [2]). 
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